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ABSTRACT 



Hawaii uses the eighth edition of the Stanford Achievement 
Test (Stanford 8) to assess academic performance of the student population in 
grades 3, 6, 8, and 10. Hawaii was not included in the norming for the 
Stanford 8, neither for the national nor the Pacific norms. In this study, 
Hawaii norms were developed based on the Stanford 8 reading and mathematics 
results from 1992 to 1996 to supplement the national norms and provide an 
additional means of comparison. Hawaii reading norms were lower at every 
grade level, especially grades 3 and 8. However, local mathematics norms 
showed Hawaii students exceeding national norms in the upper quartile on 
grades 3, 6, and 8. Hawaiian and national grade-10 mathematics norms were 
very similar. Average performance changes between grades were analyzed, and 
it was found that the longitudinal cohorts made greater gains in achievement 
from third to sixth and from eighth to tenth grades than national 
counterparts, while the sixth- to-eighth grade group made lesser gains. These 
norms provide tools to improve the understanding of Hawaii student 
performance relative to their mainland counterparts. (Contains 5 tables, 8 
figures, and 26 references.) (Author/SLD) 



******************************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document . * 



******************************************************************************** 



ED 408 339 



Creating Norms for Hawaii 1 



Running head: CREATING NORMS FOR HAWAII 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

Creating Local Norms to Evaluate Students 

received from the person or organization 
originating it. 

improve reproduction quality. | in a Norm-Referenced Statewide Testing Program 



Z ATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

s document has been reproduced as 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HAS BEEN GRANTED BY 

$>£ur#-V\ S 



Points of view or opinions stated in this 

Soeri M8iti^ e or 8 ^L , ® pre8en ' Sarah S. Gronna and Amelia A. Jenkins to the educational resources 

y INFORMATION CENTER (ERIC) 

University of Hawaii at Manoa 



Selvin A. Chin-Chance 



Planning and Evaluation Branch, Hawaii Department of Education, Honolulu, Hawaii 



Sarah S. Gronna is a concurrent doctoral student in the Department of Educational 
Psychology and the Department of Special Education at the University of Hawaii, and Amelia A. 
Jenkins is an Assistant Professor in Special Education at the University of Hawaii. Selvin A. 
Chin-Chance is Administrator of the Testing Office in the Hawaii Department of Education. We 
would like to gratefully acknowledge the assistance provided by Patricia Ishimura for her helpful 
insights, advice, and patience. 

Correspondence concerning this paper should be addressed to Sarah S. Gronna, Hawaii 
Department of Education, Test Development Section, 3430 Leahi Avenue, Building D, 1 st Floor, 
Honolulu, HI 96815. Electronic mail may be sent via Internet to gronna@hawaii.edu. 



Creating Norms for Hawaii 2 



Abstract 

Hawaii uses the Stanford Achievement Test (Stanford 8) to assess academic performance 
of the student population in grades 3, 6, 8, and 10. Hawaii was not included in the norming for 
the Stanford Achievement Test 8 th edition (Stanford 8), neither for the national nor the pacific 
norms. Hawaii norms were therefore developed based on the Stanford 8 reading and mathematics 
results from 1992 to 1996 to supplement the national norms and provide an additional means of 
comparison. Hawaii reading norms were lower at every grade level, especially grades 3 and 8. 
However, local mathematics norms showed Hawaii students exceeding national norms in the 
upper quartile in grades 3, 6, and 8. Hawaii and national grade 10 mathematics norms were very 
similar. Average performance changes between tested grades were analyzed. The longitudinal 
cohorts made greater gains in achievement from 3 rd - 6 th and 8 th - 10 th grades than the national 
counterparts, while the 6 th - 8 th grade group made lesser gains. These norms provide tools to 
improve the understanding of Hawaii student performance relative to their mainland 
counterparts. 



O 

ERLC 



3 



Creating Norms for Hawaii 3 



Creating Local Norms to Evaluate Students 
in a Norm-Referenced Statewide Testing Program 
Assessment of student achievement is required by legislation in many states.The Hawaii 
Department of Education mandates annual testing for all public school students in grades 3, 6, 8, 
and 10 unless specifically excluded from testing. The most widely used standardized tests are 
norm-referenced ones, which were developed to compare individual student performance to a 
representative national sample. The norm-referenced Stanford Achievement Test, 8 th Edition 
(Stanford 8), published in 1992 by the Psychological Corporation, is generally the standardized 
instrument used to measure academic achievement in Hawaii. The Stanford 8 results can be 
reported in a variety of normed scores (e.g., stanines, percentiles, and scaled scores) for use in 
comparing students in the target group (Hawaii) with the norming group (Heim, 1994). Even 
though commentators have emphasized the shortcomings and questioned the validity of 
standardized tests, it is improbable that these tests will be discarded as assessment instruments 
(Crouse & Trusheim, 1989; Powell & Steelman, 1996) as they appear to be the most efficient, 
and relatively objective method of measuring student achievement. 

There have been numerous controversies surrounding the use of norm-referenced tests in 
Hawaii (Aizawa, 1994; Chin-Chance, 1994; Heim, 1994; Paris, 1994). However, in order to 
consider the appropriateness of administering the Stanford 8, the purpose of the mandated 
assessment must be understood: to assess the performance of students. To assess the achievement 
of Hawaii public school students, their performance on the Stanford 8 is compared with the 
performance of other students and schools in the nation. 
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The Stanford 8 national norms are based upon a sample (n=l 75,000) of students assessed 
in 1991 (The Psychological Corporation, 1992). However, students in Hawaii’s public schools 
were not included in the sample, neither in the pilot testing nor in the development of the 
national norms. The Psychological Corporation maintains that Hawaii’s lack of representation in 
the national norming process does not significantly affect the national norms, as Hawaii’s student 
population includes ethnic groups that comprise only 1-2% of the United States’ student 
population (J. Mayo, personal communication, September, 1994). Moreover, when Stanford 8 
norms were developed for the Pacific region, students in American Samoa, Guam, and CNMI 
were included but students in Hawaii were not (M. Turituri, personal communication, October 
23, 1996). 

It is often advantageous for a state or school district to develop special norms to 
supplement the national norms, which may be inadequate for the local setting, to provide an 
additional means of comparison (Nunnally, 1972; Petersen, Kolen, & Hoover, 1989; Brown & 
Bryant, 1984). Brown and Bryan recommend the development of local norms when there are 
significant differences in characteristics of ethnicity, gender, achievement performance, or age 
between the local population and the normative group. The typical characteristics of Hawaii’s 
public school students warrant the development of local norms. 

Rationale for Development of Local Norms in Hawaii. 

Ethnicity. Unlike the rest of the contiguous states, the ethnicity of the public school 
population in Hawaii is approximately 23% Caucasian, 4 % Hispanic, .4% African American, 
21% Native Hawaiian and Part Hawaiian, 34% Asian, and 18% other (Chin-Chance, Gronna, & 
Jenkins, 1996a). The Stanford 8 norms, however, were developed using a representative national 
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sample of 72% Caucasian, 7% Hispanic, 15% African American, and 5% other ethnicity (The 
Psychological Corporation, 1992b). Studies have linked achievement in Hawaii to ethnicity 
(Brandon, 1984; Educational Testing Service, 1993; Gallimore, Boggs, & Jordan, 1974). The 
national norms therefore do not reflect Hawaii’s multicultural student population and 
achievement. 

Gender. The achievement pattern in mathematics is atypical for boys and girls in Hawaii 
(Brandon, Jordan, & Higa, 1995; Kiplinger, 1996). The Total Reading and Total Mathematics 
“dimensions” of the Stanford 8 are computed based on the scores of several subtests in each area. 
Hawaii is the only large school district in the United States where girls outperform boys in 
Stanford 8 Total Reading and Total Mathematics at all grade levels (Liskum & Chin-Chance, 
1996). 

Achievement. Unlike the national average (10.4%), Hawaii has a higher (19%), 
proportion of students enrolled in private schools (Lai, Saka, & Chin-Chance, 1994). Research 
indicates that private school students who once attended public schools in Hawaii typically score 
in the above average range of achievement (Lai, et al., 1994). Furthermore, 12.9 % of the tested 
student population in Hawaii is comprised of special education students (Chin-Chance, Gronna, 
& Jenkins, 1996b). Studies have indicated that students with disabilities typically score in the 
below average range of achievement (Gronna, Jenkins, & Chin-Chance, 1996). The Stanford 8 
normative sample only includes 4.9% special education students (The Psychological 
Corporation, 1992a). When Hawaii is compared to the nation, the normative sample includes 
more “bright” students who are not attending private schools and fewer students with disabilities. 
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Age. Hawaii is one of eight states maintaining late entrance admission (December 1 to 
January 1) cut-off dates (Liskum & Chin-Chance, 1996). Hawaii has more “younger” students 
within each grade than other states. Younger children are usually at an academic disadvantage 
when compared to older classmates (Crosser, 1991). Liskum and Chin-Chance found a 
relationship between age and Stanford 8 test scores in Hawaii at all tested grades. Children bom 
in the last three months of the year, had statistically significant lower achievement scores than 
their older peers. Liskum and Chin-Chance (1996) suggest that Hawaii is at relative disadvantage 
in norm-referenced comparisons with other states, because the Stanford 8 normative sample was 
based on an older population. 

Purpose of study 

This study sought to analyze the standardized test score data for students within Hawaii 
in order to profile their achievement. We hypothesized that the overall test performance of 
students within Hawaii is different from the mainland population. This study developed local or 
state-wide norms for public school students taking the Stanford 8 in grades 3, 6, 8, and 10 based 
on cross-sectional data from 1992 to 1996 in order to assess performance of these students. A 
longitudinal analysis was conducted to develop a sense of typical changes in reading and 
mathematics based on the Stanford 8. 

Method 

The Hawaii Department of Education, Test Development Section of the Planning and 
Evaluation Group maintains an extensive student identity database on all public school students 
that includes ethnicity, home language, age, and gender. The Stanford 8 results have been stored 
in annual databases. The student identity and Stanford 8 databases for 1 992 through 1 996 (n = 
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247,817) were used for analysis. During this period, approximately 68,679 grade 3 students, 
66,553 grade 6 students, 60,400 grade 8 students, and 51,185 grade 10 students were assessed 
using the Stanford 8. 

Procedure 

Development of Hawaii Norms based on Cross-Sectional Data. Using Microsoft Access 
(Microsoft, 1995) the individual student records (N — 1,069,500) in the 1992 to 1996 
demographic databases were disaggregated into tested grades and linked by student identification 
numbers to the Stanford 8 databases. The cross-sectional data from 1992 and 1996 were 
combined to provide more stable benchmarks. Of these, all Total Reading student scaled scores 
and Total Mathematics scaled scores in grades 3, 6, 8, and 10 were used to develop Hawaii 
norms for the test dimensions at each tested grade level. The Stanford 8 scaled scores represent 
approximately equal units in learning on a continuous scale from 1 to 999 and facilitate the 
conversion into other score types that are suitable for studying the change in performance over 
time (The Psychological Corporation, 1992a). Descriptive statistics and frequency distributions 
of scores within tested grades were calculated using SPSS 7.0 for Windows 95 (SPSS, 1996). 
From these data, tentative Hawaii percentile rank norms were constructed. 

National normative data were compiled from the technical data reports for the Stanford 8 
(The Psychological Corporation, 1 992). The normative data for scaled scores and percentiles 
were published in the “look-up” tables by administration data and test form. The means and 
standard deviations of the scaled scores of the combined spring standardization sample and 
number of students included in the sample were published in National technical reports (The 
Psychological Corporation, 1992). The mean scores for the combined Spring standardization 
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sample may vary from specific values found in the “look-up” tables due to slight differences 
between test forms. The mean scaled scores for Total Reading and Total Mathematics 
dimensions were determined from the Stanford 8 technical reports and were identified as the 
nationally normed scaled score at the 50th percentile rank for each grade. The arithmetic 
differences between national means at sequential grades were computed to determine the national 
average scaled score increases per grade level and test dimension. 

Changes in Scaled Scores Between Grades Based on Longitudinal Data. To develop 
longitudinal cohorts, the student identity and Stanford 8 data bases were further linked to test 
results at an earlier grade. The longitudinal analysis was based on two 3rd to 6th grade cohorts (n 
— 20,826). Cohort A included 3rd grade students in 1992 and 6th grade students in 1995 (na = 
10,429). Cohort B included 3rd grade students in 1993 and 6th grade students in 1996 (nb = 
10,397). Three 6th to 8th grade cohorts (n = 29,948) were included. Cohort C included 6th grade 
students in 1992 and 8th grade students in 1994 (nc = 9,781). Cohort D included 6th grade 
students in 1 993 and 8th grade students in 1995 (nd = 9,980). Cohort E included 6th grade 
students in 1994 and 8th grade students in 1996 (ne =10,187). Finally, three 8th to 10th grade 
cohorts were included in the longitudinal analysis (n_= 23,683). Cohort F included 8th grade 
students in 1992 and 10th grade students in 1994 (nf = 7,886). Cohort G included 8th grade 
students in 1993 and 10th grade students in 1995 (ng = 7,823). Cohort H included 8th grade 
students in 1994 and 10th grade students in 1996 (nh = 7,974). 

The cohorts data were aggregated by grade grouping to provide more stable benchmarks. 
To study the changes between the tested grades the Psychological Corporation scaled scores were 
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used as they adequately represent approximately equal units on a continuous scale and are 
equivalent across test forms and test levels (The Psychological Corporation, 1992b). 

Hawaii scaled score mean increases were determined for each longitudinal cohort’s Total 
Reading and Total Mathematics dimensions. The associated scaled score corresponding to the 
50th percentile rank of the frequency distribution was identified at each grade level and 
dimension. The arithmetic difference between Hawaii means at sequential grades were computed 
to determine the Hawaii average scaled score increases per cohort and test dimension. 

Results 

Tables 1 and 2 represents the descriptive statistics, means, and standard deviations for 
Stanford 8 Total Reading and Total Mathematics scaled scores for each Hawaii grade level based 
on the cross-sectional data. Hawaii means and standard deviations for test dimensions are 
different from the national norms at each grade level. The Total Mathematics kurtosis (.910 to - 
.145) and skewness (.817 to .390) and Total Reading kurtosis (-.405 to .116) and skewness (.293 
to .468) for each Hawaii frequency distribution of scores are represented by grade in Table 3. The 
distribution of scores is generally positively skewed. The distribution of the Hawaii scaled scores 
does not fall along the normal distribution of the Stanford 8 scaled scores for the Total Reading 
and Total Mathematics dimensions. 

Tentative Hawaii percentile rank norms were constructed for grades 3, 6, 8, and 10 from 
the scores of all students who took the Stanford 8 in a standardized manner. These norms include 
students with disabilities and have a fairly equal representation of male and female students (e.g., 
males: 51%, 50.5%, 51.2% and 50.1% for grades 3, 6, 8, and 10, respectively). Figures 1 through 
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8 compare Hawaii percentile rank norms to national norms for reading and mathematics 
dimensions. 

The reading performance gap between the Hawaii and national norms is the greatest at 
grades 3 (14 to 20 scaled score points) and 8 (10 to 16 scaled score points). Smaller performance 
gaps are noted for grades 6 and 1 0. The performance gap is not consistent across all performance 
levels. For grades 3, 6, and 10 the largest differences occur in the mid-range while grade 8 
Hawaii students demonstrate the most severe gap in the lower half of the performance range. 

Hawaii mathematics performance indicates relatively small differences (3 to 6 scaled 
score points) when compared to national norms. In grades 3, 6, and 8, Hawaii students perform 
better than their national counterparts in the upper quartile range and less well in the middle 
quartiles. At grade 10, the national normative group performs better at almost every level, but 
only at a very small amount (3 to 4 scales score points). 

Average scaled score values were further analyzed to ascertain the average changes in 
scaled scores between grades based on a longitudinal analysis the data. Between 3 rd and 6 th 
grades the Hawaii group increased an average of 1 1 Total Reading and 14 Total Mathematics 
scaled score points above the national norm. However, between 6 th and 8 th grades, the Hawaii 
group gained 2 scaled score points less in Total Reading and 16 points less in Total Mathematics 
dimensions. Between 8 th and 10 th grade, Hawaii students increased an average of two scaled score 
points above the national group in Total Reading and exhibited equal scaled score increases in 
Total Mathematics when compared to the gains made nationally (see Tables 4 and 5). 



Discussion 



Creating Norms for Hawaii 1 1 



The primary purpose of this study was to analyze the standardized test score data for 
students within Hawaii in order to evaluate academic performance within the state. Statewide 
norms for Hawaii’s public school students were developed by grade level in order to provide an 
indicator of Hawaii “normal” performance. By developing local norms, students in Hawaii are 
being compared to the performance of a more relevant heterogeneous population. 

Closer inspection of the Hawaii norms reveal that at all four grades tested, Hawaii 
students do not perform on par with the national normative group. Hawaii students fail to 
perform on par in reading with their national counterparts, yet are relatively equivalent in 
mathematics performance. Hawaii reading norms were lower at every grade level, especially 
grades 3 and 8. However, Hawaii mathematics norms showed Hawaii students exceeding 
national norms in the upper quartile in grade 3, 6, and 8. Hawaii and national grade 10 
mathematics norms were very similar. In mathematics, Hawaii students generally perform much 
closer to their national counterparts. 

The longitudinal cohorts made greater gains in achievement from 3 rd - 6 th and 8 th -10 th 
grades than their national counterparts, while the 6 th - 8 th grade group made lesser gains. It is 
interesting that while gains between grade 6 and 8 are not as substantial as the national normative 
gains between these years, the grade 3 and 6 cohorts as well as grade 8 and 10 cohorts increase or 
remain equivalent to national normative performance achievement. These findings are 
compatible with previous research (Chin-Chance, et al., 1996a; Gronna, et al., 1996; Lai, et al., 

1 994). The variables that can account for the increase or decrease in Stanford 8 reading scores 
have not yet been identified nor evaluated. Perhaps the differences in scores between grades 
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could be related to school philosophies, perceived self-efficacy, students’ background 
characteristics, motivation, or gender. 

Since the results of our study are based on the entire public school population at targeted 
grades, the differences reported must be viewed in terms of educational significance rather than 
statistical significance. Undoubtedly the grade 3 to 6 cohort differences are large enough to 
represent relative large changes in relative rankings between Hawaii and national norms while 
the two to three point differences in the grade 8 to 10 cohort would represent relative minor 
changes in relative rankings. These analysis and norms provide additional tools to improve the 
understanding of Hawaii student performance relative to their mainland counterparts. 

Summary 

There are four important features to use when examining norms: (a) the types of derived 
scores that are reported, (b) the demographic representativeness of the normative sample, (c) the 
size of the normative group, and (d) the recency of test standardization (Wallace, Larsen, & 
Elskin, 1992). One must be wary of comparing the results of students in Hawaii, who take the 
Stanford 8, to the national norm. The Hawaii “norms” developed in this study more accurately 
reflect the demographic student characteristics of Hawaii’s public school students than the 
national or Pacific region norms. The Hawaii norms are based on a population much larger than 
the Stanford 8 sample, and are more current. 

The Hawaii norms can provide additional information about how well individual students 
are performing in comparison to students who are similar to them on important characteristics 
such as gender, ethnicity, and age. The Hawaii norms could be used to further identify the 
influence of gender and age on achievement measured by the Stanford 8. The possible finding 
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could influence school entrance age requirements, curriculum, and teaching styles. The Hawaii 
norms could be additionally compared to the Stanford 8 norms of the Pacific region for further 
evaluation of achievement within the Pacific basin. Use of these norms do not imply that the 
more traditional norms are incorrect. It should be viewed from the standpoint that these norms 
provide additional interpretive tools for understanding students performance in Hawaii. 
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Table 1 



Total Reading Scaled Scores for Stanford Achievement Test 8: 1992- 1996 





M 


Hawaii 

SD 


n 


SD 


Natio 

n 

M 


n 


Third 


590 


38 


65,866 


40 


611 


9,617 


Sixth 


642 


34 


65,218 


38 


656 


9,870 


Eighth 


661 


36 


58,033 


36 


676 


8,711 


Tenth 


679 


35 


47,157 


36 


688 


6,491 



Note. The mean scores for the combined Spring standardization sample may vary from specific 
values found in the “look-up” tables due to slight differences between test forms. 



Creating Norms for Hawaii 1 9 



Table 2 




Total Mathematics Scaled Scores for Stanford Achievement Test 8: 1992- 1996 





M 


Hawaii 

SD 


n 


SD 


Natio 

n 

M 


n 


Third 


596 


43 


66,623 


40 


596 


9,636 


Sixth 


657 


37 


64,978 


39 


663 


9,792 


Eighth 


683 


39 


57,473 


39 


690 


8,671 


Tenth 


707 


44 


48,028 


44 


705 


6,440 



Note. The mean scores for the combined 



values found in the “look-up” tables due 



Spring standardization sample may vary from specific 
to slight differences between test forms. 
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Table 3 

Kurtosis and Skewness for Frequency Distributions of Scaled Scores in Hawaii 



Total Reading Total Mathematics 



Grade 


Kurtosis 


Skewness 


Kurtosis 


Skewness 


Third 


-.407 


.334 


-.145 


.390 


Sixth 


-.193 


.333 


.282 


.656 


Eighth 


.116 


.468 


.588 


.817 


Tenth 


-.225 


.293 


.910 


.801 
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Table 4 

Mean Total Reading Scaled Score Differences for Selected Cohorts. 





Grade 3 to 6 




Grade 6 to 8 






Grade 8 to 10 






3 


6 


A 


6 


8 


A 


8 


10 


A 


Hawaii 


591 


643 


52 


640 


662 


22 


666 


680 


14 


National sample (Typical) 


608 


649 


41 


649 


673 


24 


673 


685 


12 





22 
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Table 5 

Mean Total Mathematics Scaled Score Differences for Selected Cohorts. 





Grade 3 to 6 


Grade 6 to 8 






Grade 8 to 10 






3 


6 A 


6 


8 


A 


8 


10 


A 


Hawaii 


598 


658 60 


657 


685 


28 


690 


710 


20 


National sample (Typical) 


594 


640 46 


640 


684 


44 


684 


704 


20 
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Figure 1. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for third 
grade Total Mathematics. 

Figure 2. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for third 
grade Total Reading. 

Figure 3. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for sixth 
grade Total Mathematics. 

Figure 4. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for sixth 
grade Total Reading. 

Figure 5. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for eighth 
grade Total Mathematics. 

Figure 6. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for eighth 
grade Total Reading. 

Figure 7. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for tenth 
grade Total Mathematics. 

Figure 8. Comparison of Hawaii (local) and national scaled scores of the Stanford 8 for tenth 



grade Total Reading. 
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